System and Method of Biomechanical Posture Detection and Feedback

ABSTRACT

A system and method are described herein for a sensor device which biomechanically detects in real-time a user&#39;s movement state and posture and then provides real-time feedback to the user based on the user&#39;s real-time posture. The feedback is provided through immediate sensory feedback through the sensor device (e.g., a sound or vibration) as well as through an avatar within an associated application with which the sensor device communicates.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of U.S. Provisional Patent ApplicationNo. 61/507,514, entitled “Biomechanics Sensory System” and filed Jul.13, 2011, the disclosure of which is incorporated herein by referenceand U.S. Provisional Patent Application No. 61/547,590, entitled“Biomechanical Sensor System and Method” and filed Oct. 14, 2011, thedisclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to biomechanical sensors usedfor biofeedback, and in particular, to biomechanical sensors thatinterface with computing devices.

2. Description of the Prior Art

Inactivity and poor posture can cause significant back problems. Aboutnine out of ten adults experience back pain at some point in their life,and five out of ten working adults have back pain every year. A. T.Patel and A. A. Ogle (2000), “Diagnosis and Management of Acute Low BackPain”, in American Family Physician, March 15; 61(6): 1779-1786.

Back pain also exacts a huge economic cost—estimated to be over $50billion per year in direct (e.g., health care expenditures) and indirect(e.g., lost productivity) costs. In the United States, back pain is theprimary reason for job-related disability, one of the leading causes ofmissed work days, the primary reason for people younger than 45 to limittheir activity, and the second most common reason for visits to thedoctor.

Medical practitioners agree that back pain can often be reduced bymaintaining good posture—that is, a “neutral spine” posture—whilesitting, standing, lying, or moving. The neutral spine posture in ahealthy spine is an optimal position of three natural curves of thespine: a neck (cervical) region, a mid-back (thoracic) region, and alower-back (lumbar) region. As shown in FIG. 1, the neck region curveinvolves cervical vertebrae C1-C7, and is ideally anteriorly convex(i.e., should curve inward). The mid-back region curve involves thoracicvertebrae T1-T12, and is ideally posteriorly convex (i.e., should curveoutward). The lower-back region curve involves lumber vertebrae L1-L5and is ideally anteriorly convex (i.e., should curve inward).

Posture is determined to some degree by pelvic tilt. As shown in diagram201 of FIG. 2, a neutral pelvis (i.e., a pelvis that is not tilted)indicates that anterior superior iliac spines (ASIS) and the pubicsymphysis (not visible in FIG. 2) fall along a vertical line runningthrough the leg bones, the anterior curves of the lumbar and cervicalregions, and out the top of the head.

If the ASIS and pubic symphysis are not properly aligned, the pelvis isrotated and thereby changes the curvatures of the lumbar, thoracic,and/or cervical regions of the spine. When the pelvis is rotated to aforward tilting position (i.e., an anteverted position), the lumbarcurvature increases beyond a neutral spine position. When this anteriorrotation of the pelvis is extreme, it can result in an unhealthyswayback posture (i.e., lordosis) as shown in diagram 202. If anindividual instead slouches, the pelvis rotates to a backward tiltingposition (i.e., a retroverted position as shown in diagram 203), therebyflattening out the cervical, thoracic, and lumbar curvatures as alsoshown in the figure.

Biofeedback (i.e., a process of using physiological indicators of abiological system to monitor and control behavior of that biologicalsystem) is one way to provide feedback to individuals about how theirbodies are functioning, with a goal of using that information to monitorand change problematic behaviors, such as smoking and weight gain. As anexample, U.S. Pat. No. 5,749,838 (Kline) discloses a posture trainingdevice which can be held against the backbone of a person by a belt orgarment such that an increase in curvature of the backbone in either amidsagittal or transverse plane moves a module within a frame relativeto the frame. When the module has been moved sufficiently to indicate apoor posture position, a tactile signal generator within the module isactivated to vibrate, thereby alerting the person that he has assumed apoor posture.

This and other conventional biomechanical devices typically requiregarments or belts to hold sensors in place. It can be difficult to applythese devices in a way that people find visually appealing, comfortable,and convenient. Furthermore, these devices tend to be bulky, adding tothe discomfort of wearing these devices. What is needed is a flexible,comfortable device that can measure a user's positions and movements toprovide postural feedback.

SUMMARY

In one embodiment is a method of providing postural feedback comprising:receiving by a microprocessor at repeated intervals data from atri-axial accelerometer, the microprocessor and the tri-axialaccelerometer comprising a sensor device attached to a user, the sensordevice further comprising memory, an actuator, and a power source;normalizing by the microprocessor the received accelerometer data;determining by the microprocessor a postural description of the userbased on the normalized received accelerometer data: and triggering bythe microprocessor the actuator to output sensory feedback based on thepostural description of the user.

In another embodiment is a postural feedback apparatus comprising: asensor device configured to be attached on a user, the sensor devicecomprising: a tri-axial accelerometer; an actuator; and a microprocessorconfigured to receive data from the tri-axial accelerometer aboutmovement of the user; normalize the received accelerometer data;determine a postural description of the user based on the normalizedreceived accelerometer data; and trigger the actuator to output sensoryfeedback based on the postural description of the user.

A non-transitory computer readable medium having stored thereuponcomputing instructions comprising: a code segment to receive by amicroprocessor at repeated intervals data from a tri-axialaccelerometer, the microprocessor and the tri-axial accelerometercomprising a sensor device attached to a user, the sensor device furthercomprising memory, an actuator, and a power source; a code segment tonormalize by the microprocessor the received accelerometer data; a codesegment to determine by the microprocessor a postural description of theuser based on the normalized received accelerometer data; and a codesegment to trigger by the microprocessor the actuator to output sensoryfeedback based on the postural description of the user.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating curvature in cervical, thoracic, andlumbar regions of a human spine.

FIG. 2 is a set of diagrams illustrating good posture and poor posture.

FIG. 3 is a block diagram illustrating a biomechanical posture detectionand feedback system according to one embodiment.

FIG. 4 is a block diagram illustrating a sensor device of abiomechanical posture detection and feedback system according to oneembodiment.

FIG. 5 is a diagram depicting appropriate placement of a sensor deviceaccording to one embodiment.

FIG. 6 is a flowchart detailing a method of biomechanical posturedetection and feedback system according to one embodiment.

FIG. 7 is a state diagram illustrating various movement states andtransitions between those movement states according to one embodiment.

FIG. 8 illustrates an exemplar acceleration pattern of a usertransitioning from a standing state to a sitting state.

FIG. 9 is a flowchart detailing steps of a rule-based model to determinemovement state according to one embodiment.

FIG. 10 is a flowchart detailing steps of a statistical model todetermine movement state according to one embodiment.

FIG. 11 is an exemplar mixture model for exemplar data clustersaccording to one embodiment.

FIG. 12 is an illustration of exemplar estimated cluster distributionhistograms for a walking state according to one embodiment.

FIG. 13 is an illustration of exemplar estimated cluster distributionhistograms for a running state according to one embodiment.

FIG. 14 is a graphical display of exemplary avatars used to providepostural feedback through a computing device to a user.

DETAILED DESCRIPTION OF THE INVENTION

Presented herein are various embodiments of a sensor device and methodsfor using the sensor device that biomechanically detects a user'sposture and provides sensory feedback in real-time about the posture tothe user. The feedback is provided in real-time through an actuator thatprovides sensory activity (e.g., vibration or sound) and/or through anavatar within a corresponding application with which the sensor devicecommunicates.

The sensor device itself comprises an accelerometer and a microprocessorthat analyzes directional movement data from the accelerometer. When thesensor device is attached to a user, the microprocessor can analyzeaccelerometer data indicative of a user's motion in real-time todetermine a user's movement state (e.g., standing, sitting, lying,running, or walking) and assess the user's posture in that movementstate (i.e., a posture quality of the user). Using various models, themicroprocessor can rapidly determine a user's movement state directly byidentifying a user's movement state or indirectly by identifyingtransitions (“state transitions”) between movement states. Thus, themicroprocessor can provide rapid feedback to the user about his/herposture.

Location of the microprocessor on the patch offers advantages overcurrent feedback devices. For example, known feedback devices thatcommunicate wirelessly with feedback display devices (e.g.,wristwatches) depend upon the wireless communication in order to makeany determinations and provide feedback, and thereby deplete batterypower more rapidly than embodiments of the sensor device discussedherein which allow continuous (24 hours a day, 7 days a week) datagathering while minimizing the energy cost of wireless communication.Unlike the sensor device discussed herein, known feedback devices (e.g.,the Polar heart monitor by Polar Electro, Finland) use a simplebroadcast model to send raw data to be interpreted by a receiving unit.Other known feedback devices perform minimal local processing (e.g.,Fitbit, by Fitbit Inc., San Francisco, Calif. and Lark, by LarkTechnologies, Mountain View, Calif.) whereas the sensor device describedherein performs local data processing and analysis in real-time.

A block diagram of the system for biomechanical posture detection andsensory feedback according to one embodiment is presented in FIG. 3.Sensor device 301 senses position and movement data over time for a user(not shown) to whom sensor device 301 is attached, processes andanalyzes the position and movement data to determine in real-time apostural description of the user, and stores the position and movementdata and postural description in memory. In some embodiments, sensordevice 301 transmits the position and movement data and posturaldescription wirelessly through an electronic communication connection302 to computing device 303 for use by a corresponding applicationrunning on computing device 303 is active. Sensor device 301 and/orcomputing device 303 can also transmit the data and postural descriptionthrough network connection 304 to a cloud server 305 on which the usermay have a personal data hub (discussed in further detail herein).

Computing device 303 is a computing device such as a smartphone (e.g.,an iPhone from Apple, Inc., a BlackBerry device from Research in MotionLimited, or a phone running the Android OS from Google, Inc. of MountainView, Calif.) or any computing device with a user interface such as aphone, personal digital assistant (PDA), media device (such as an iPodor iPod Touch from Apple, Inc.), electronic tablet (such as an iPad fromApple, Inc., or an HP Slate from Hewlett-Packard Development Company,L.P.), an electronic reader device (such as a Kindle or Kindle DX fromAmazon.com, Inc. of Seattle, Wash., or The Reader from SONY ElectronicsInc.), a laptop computer, or a desktop computer.

One of ordinary skill in the art will understand that electroniccommunication connection 302 between sensor device 301 and computingdevice 303 can be, but need not be the same as network 304 betweencomputing device 303 and cloud 304. Electronic communication connection302 is a wireless computing (e.g., governed by Wi-Fi protocols such asBluetooth) or telecommunications (e.g., governed by 3G or 4G protocols)connection or network. Network 304 can be a wireless or wired network,or a telecommunications connection or network (e.g., governed by 3G or4G protocols). Electronic communication connection 302 and network 304can each or both be a wide area network (WANs), a local area network(LAN), a global area network (GAN), a virtual private network (VPN), apersonal area network (PAN), an enterprise private network, or anysimilar network now known or later developed.

Embodiments of sensor device 301 have a variety of form factors. Oneembodiment of sensor device 301 is shown in the block diagram of FIG. 4.In this embodiment, sensor device 301 comprises a sensor circuit board402 attached to a pad, which pad is attached to a thin and flexibleadhesive. The flexible adhesive can be any removable adhesive that willstick to human skin and/or cloth (e.g., nylon, polyester). In anotherembodiment, sensor device 301 comprises an attachable or clip-on devicein addition to or instead of the flexible adhesive. Such a device (notshown) can be used to attach sensor device 301 to, e.g., a belt,clothing, sports equipment, or a carrying bag. In yet anotherembodiment, sensor device 301 can be embedded into other form factorssuch as a belt, a tight-fitting athletic shirt, a band, or underwear.

In one embodiment, sensor circuit board 402 comprises a microprocessor403 with memory, a wireless communication module 404, an accelerometer405, an actuator 406, and a power source 407 (e.g., battery) whichpowers sensor circuit board 402. In other embodiments, the memory (e.g.,a memory chip) is external to microprocessor 403. In other embodiments,sensor circuit board 402 does not comprise actuator 406.

Accelerometer 405 is a tri-axial accelerometer configured to measureproper acceleration (i.e., experienced acceleration relative tofree-fall) along horizontal (x-), vertical (y-), and transverse(z)-axes. As the user (with sensor device 301 attached) moves,accelerometer 405 captures tri-axial acceleration data (i.e., data alongthe x-, y-, and z-axes) at, e.g., a 50 Hz sampling frequency. Thesampled data are passed to microprocessor 403 for storage and forprocessing to determine a postural description of the user. A posturaldescription comprises a user's movement state (e.g., standing, sitting,lying, running, or walking) and/or posture quality (i.e., how well theuser is balanced within that movement state, e.g., whether and how wellthe user's posture matches a desirable “good posture” for the determinedmovement state). Microprocessor 403 then triggers actuator 406 toprovide sensory feedback to the user in real-time based on thedetermined postural description of the user.

Microprocessor 403 is configured for cycling at a given rate, preferablyprocessing accelerometer data at a sample rate of 32 Hz. Microprocessor403 uses various models discussed herein (e.g., a rule-based model, astatistical model, a library comparison model, and/or a tilt anglemodel) to analyze accelerometer data in order to determine user movementstates and posture quality. Microprocessor 403 is further configuredwith memory to store data received from accelerometer 405 as well asdetermined movement and posture quality. Microprocessor 403 isadditionally configured to use movement and posture quality to commandfeedback based on the determined movement states and posture quality tobe delivered through actuator 406 and/or through computing device 303 tothe user.

Actuator 406 is a feedback delivery apparatus such as a vibratory motorand/or a sound device. In other embodiments, computing device 303 can bean additional feedback delivery apparatus and/or can be used as afeedback delivery apparatus in lieu of actuator 405.

Microprocessor 403 forwards the postural description determinationsand/or the accelerometer data, to wireless communication module 404which in turn communicates wirelessly with a corresponding applicationon computing device 303 to synchronize the data and postural descriptioninformation with a user's personal data hub (or profile) stored oncomputing device 303 or on cloud server 305. Microprocessor 403 alsostores the accelerometer data in memory if the sensor device is not inrange for communication with computing device 303.

Microprocessor 403 synchronizes through wireless communication module404 with the application running on computing device 303 and/or withcloud server 305 to upload the user's data and determined movementstates. The rapid processing and storage capabilities of microprocessor403 offer benefits over known feedback devices (e.g., feedback sensorscommunicating with a user's wristwatch to provide feedback).

Wireless communication module 404 is configured to communicatewirelessly with computing device 303 via a network protocol such as aBluetooth proprietary open standard (created by Ericsson of Sweden andmanaged by the Bluetooth Special Interest Group, Kirkland, Wash.), orthrough a cellular network. Wireless communication module 404synchronizes user data (e.g., postural descriptions, pelvic tilt angles,determined movement states, state changes, and/or raw accelerometerdata) through electronic communication connection 302 with computingdevice 303 for use by a corresponding application running on computingdevice 303. Computing device 303 synchronizes user data through networkconnection 304 with a user's personal hub in cloud server 305.

In other embodiments, one or more additional sensor can be added tosensor device 301 to enhance accuracy of posture and movement detection.Examples include other light weight, very thin and/or unobtrusivemicroelectrical mechanical system (MEMS)-based sensors. In oneembodiment, a MEMS-based gyroscope can be added to sensor device 301 toenhance rotary sensing capability (i.e., ability to detect twisting andbending). In another embodiment, a MEMS-based strain gauge can be usedto determine a relational change between the pelvis and the lower back.The relational change alters resistance of the strain gauge, whichresistance can be mapped to various states of good and bad posture. Inyet another embodiment, MEMS-based electromyogram sensors can be used todetect muscle strain which can be correlated to various states ofposture.

Referring now to FIG. 5, sensor device 301 can be attached to a body invarious locations. For example, as indicated, sensor device 301 can beattached to a user's back (e.g., in a belt or band embodiment) at thelevel of the L4-L5 vertebrae (i.e., at the pelvis just below the waist)and/or attached to a user's back (e.g., in an adhesive patch embodiment)at the level of the S1-S2 vertebrae (i.e., at the pelvis a few inchesbelow the waist), and/or attached to the user's neck at the level of theC4-C5 vertebrae (i.e., on the back of the neck a few inches from thebase of the head). Once sensor device 301 is attached, many variablescan affect data obtained from accelerometer 405. For example,differences in user body characteristics, belt models, shoes,manufacturing tolerances, etc. can cause variability in theaccelerometer measurements by altering alignment of the accelerometeraxes with the three-dimensional axes in reality (e.g., high heels maytilt a user's spine forward such that the y-axis of the user measured byaccelerometer 405 is no longer a truly vertical y-axis in space).Additionally, the user might apply the sensor in an upside-downorientation. Or, orientation of accelerometer 405 on sensor circuitboard 402 can change between product releases.

Effects of these and other factors can be minimized with a procedure tonormalize accelerometer measurements so that the measurements fromaccelerometer 405 indicating acceleration along the x-, y-, and/orz-axes correspond to movement of body position along those axes.Accelerometer 405 can be imagined as having three perpendicular axes,with weights on springs sliding along the axes. The x and z weights arein the middle of their respective axes, representing an accelerometerreading of x=0 and z=0. Gravity pushes down the weight on the y-axis,resulting in a reading of 1G for y.

To control for these effects, in one embodiment, accelerometer 405 iscalibrated and normalized so as to align the x-, y-, and z-axes to acorrect orientation wherein accelerometer data deviations from zero onthe x-axis indicate a lateral (side-to-side) movement of the user,accelerometer data deviations from zero on the y-axis indicate avertical (up and down) movement of the user, and accelerometer datadeviations from zero on the z-axis indicate a transverse (rocking, e.g.,dorsal-to-ventral) movement of the user.

If the accelerometer is normalized correctly, horizontal movement (e.g.,leaning left or right) should result in non-zero x-values as theimagined x-weight slides away from the point where the axes meet andforwards or backwards movement (e.g., leaning forwards or backwards)should result in non-zero accelerometer z-values as the imaginedz-weight slides away from the point where the axes meet. Since gravitynormally slides the imagined weight along the y-axis, free-fall wouldresult in y=0 (followed by a large value for y when the user hits theground).

Sensor Orientation Normalization.

One embodiment of normalization will now be discussed in more detail. Aknown difficulty with using an accelerometer on a user is that the threeaxes are rotated in some unknown way because of the variables discussedabove. Thus, a rotation matrix R₀ is used to align the accelerometerwith gravity such that

${R_{0}\begin{pmatrix}x_{0} \\y_{0} \\z_{0}\end{pmatrix}} = \begin{pmatrix}0 \\1 \\0\end{pmatrix}$

where (x₀, y₀, z₀) is a first accelerometer measurement when the user isstanding with good posture.

Unfortunately, the single rotation matrix R₀ will not necessarilyconform to the requirement that z should remain 0 if the user leansright, and that x should remain 0 if the user leans back. And even ifthese requirements are met, the signs of the measurements could still beinverted.

To deal with this issue, a second measurement (x₁, y₁, z₁) is taken whenthe user is sitting in a chair and leaning back. After putting thismeasurement vector through the rotation R₀, a second rotation R₁ aroundthe y-axis is used to align the accelerometer with a y-plane of the x,y, z coordinate system such that

${R_{1}{R_{0}\begin{pmatrix}x_{1} \\y_{1} \\z_{1}\end{pmatrix}}} = \begin{pmatrix}0 \\{\overset{\sim}{y}}_{1} \\{\overset{\sim}{z}}_{1}\end{pmatrix}$

where {tilde over (y)}₁ is the y-axis coordinate after two rotations and{tilde over (z)}₁ is the z-axis coordinate after two rotations. BecauseR₁ rotates around y, the axial matrix is still

${R_{1}{R_{0}\begin{pmatrix}x_{0} \\y_{0} \\z_{0}\end{pmatrix}}} = \begin{pmatrix}0 \\1 \\0\end{pmatrix}$

The orientation matrices R₀ and R₁ are computed at least once for a userand can then be stored in memory and reused. If the user wears thesensor in a different way, or switches to a substantially different pairof shoes, a new set of matrices is needed, so the user recalibrates andrenormalizes sensor device 301.

One challenge is how to get the user to stand with good posture in orderto take baseline accelerometer measurements (x₀, y₀, z₀). Explaininggood posture to each user so s/he can provide appropriate input (i.e.,move appropriately) to make sensor device 301 work in a meaningful wayis impractical. Fortunately, an average measurement (x _(w), y _(w), z_(w)) of a walking user provides an excellent approximation of theneeded good posture measurement (x₀, y₀, z₀). Thus, the user isinstructed to walk naturally for about twenty seconds while wearingsensor device 301, during which period microprocessor 403 records anumber of measurements (e.g., 750 measurements) from accelerometer 405.The average of the sampled walking measurements is then used to computeR₀.

To increase the quality of the samples, microprocessor 403 applies asimple motion detection procedure (which works regardless of sensororientation) to identify aberrant measurements that were obviously notproduced by a walking user. In one embodiment, accelerometer data(x_(t), y_(t), z_(t)) are collected for every time t (e.g., at 32 Hz).Power at each time point (A_(t)) is computed using the equation

A _(t)=ln(x _(t) ² +y _(t) ² +z _(t) ²).

Power is compared over successive data points. A change in power (D_(t))over successive data points (A_(t) and A_(t-1)) indicates movement. Thetime series D_(t) is then smoothed using the equation

$S_{t} = {{32\mspace{11mu} D_{t}} + {\sum\limits_{k = 1}^{31}\; {\left( {32 - k} \right)\mspace{11mu} \left( {D_{t + k} + D_{t - k}} \right)}}}$

where S is the smoothed time series, D is a change in power, t is a timeindex, and k is a looping index. S_(t) is then compared to definedthresholds to determine whether the subject is walking or sitting attime t. For example, if the accelerometer is configured such thatgravity is represented by a value of 248, the threshold to identify awalking state is, e.g., 160 and the threshold to identify a sittingstate is, e.g., 45. Thus, if S_(t) is calculated to be 170 (i.e.,greater than the walking threshold), a walking state is detected,whereas if S_(t) is calculated to be 40 (i.e., less than the sittingthreshold), a sitting state is detected. If S_(t) values range betweenmovement state thresholds (e.g., between 45 and 160 in this example),the sample is discarded due to classification uncertainty. In this way,aberrant measurements can be rejected from the sampled measurementsbefore the average is computed.

The second measurement (x₁, y₁, z₁) is easier to acquire. The user isinstructed to sit in a chair and lean back. Although the user might leanback more or less than desired for good posture (e.g., depending on thechair or on personal preference), any such deviation does not impact R₁directly. Leaning further back in the chair helps to improve accuracy ofrotation, but does not impact an angle of rotation per se.

Calculation of R₀ and R₁.

Known algorithms can be used to compute the rotation matrices. Apreferred process is presented below.

The first goal is to find a matrix R₀ such that

${R_{0}\begin{pmatrix}x_{0} \\y_{0} \\z_{0}\end{pmatrix}} = {{R_{0}v_{0}} = \begin{pmatrix}0 \\1 \\0\end{pmatrix}}$

where ν₀ is the vector derived by averaging the walking samples. Anassumption is made that that |ν₀|=1, which assumption is plausiblebecause gravity is, by definition, 1G and all walking movementscompensate each other. Experimentation indicates that the assumption isvalid.

In a first step, ν₀ is rotated around the z-axis by a rotation R_(z)until the x-coordinate disappears (i.e., x becomes 0). Then a secondrotation R_(x) around the x-axis eliminates the z coordinate (i.e., zbecomes 0) until eventually

R ₀ =R _(x) R _(z)

where

${R_{z} = \begin{pmatrix}{\cos \; \phi} & {{- \sin}\; \phi} & 0 \\{\sin \; \phi} & {\cos \; \phi} & 0 \\0 & 0 & 1\end{pmatrix}},{R_{x} = \begin{pmatrix}1 & 0 & 0 \\0 & {\cos \; \theta} & {{- \sin}\; \theta} \\0 & {\sin \; \theta} & {\cos \; \theta}\end{pmatrix}},$

angle φ is defined as

${\phi = {{{sgn}\left( x_{0} \right)}a\; {\cos\left( \frac{y_{0}}{\sqrt{x_{0}^{2} + y_{0}^{2}}} \right)}}},$

and angle θ is defined as

$\theta = {{- {{sgn}\left( z_{0} \right)}}a\; {{\cos\left( \frac{\sqrt{x_{0}^{2} + y_{0}^{2}}}{v_{0}} \right)}.}}$

The next goal is to find a matrix R₁ representing a rotation around they-axis such that

${R_{1}{R_{0}\begin{pmatrix}x_{1} \\y_{1} \\z_{1}\end{pmatrix}}} = \begin{pmatrix}0 \\{\overset{\sim}{y}}_{1} \\{\overset{\sim}{z}}_{1}\end{pmatrix}$

where (x₁, y₁, z₁) is an accelerometer measurement while the user issitting comfortably in a chair, and R₀ is the rotation matrix derivedfrom averaged accelerometer measurements while the user is walking.

R₁ can be computed as

$R_{1} = \begin{pmatrix}{\cos \; \phi} & 0 & {\sin \; \phi} \\0 & 1 & 0 \\{{- \sin}\; \phi} & 0 & {\cos \; \phi}\end{pmatrix}$

where the angle φ is defined as

$a\; {{\sin\left( \frac{x_{0}}{\sqrt{x_{0}^{2} + z_{0}^{2}}} \right)}.}$

After the data are normalized, microprocessor 403 stores the normalizeddata for analysis to determine the user's movement state and/or posturequality.

Biomechanical Posture Detection and Feedback.

A flowchart of a method of biomechanical posture detection and feedbackaccording to one embodiment is presented in FIG. 6. In step 601,microprocessor 403 receives and normalizes accelerometer data streamedfrom accelerometer 405 in real-time as discussed above. Once sensordevice 301 has been normalized, sensor device 301 is used to detectposture of a user and provide sensory feedback to that user.

In step 602, microprocessor 403 analyzes the accelerometer data todetermine the real-time movement state and/or postural state of theuser. The user's movement state (e.g., standing, sitting, lying,running, or walking, discussed further herein with respect to FIGS. 9and 10) and posture quality (i.e., how well the user is balanced withinthat movement state, e.g., whether and how well the user's posturematches a desirable “good posture” for the determined movement state)together form a postural description of the user. In variousembodiments, the user's movement state is determined by a librarycomparison model, a rule-based model, a statistical model, and/or a tiltangle model, each of which are described elsewhere herein. The tiltangle model is also used to determine a user's posture quality. Thereal-time postural description information is time-stamped and stored inmemory, and can be retrieved from memory for future analysis (e.g.,comparison to incoming accelerometer data) and/or for latersynchronization with computing device 303 and/or cloud server 305.

In step 603, microprocessor 403 triggers actuator 406 to output sensoryfeedback to the user based on the determined real-time posturaldescription of the user. Steps 601, 602, and 603 are looped continuouslyin real-time as long as the user has an attached operating sensor device301.

In optional step 604, microprocessor 403 transmits the time-stampedpostural descriptions to computing device 303 on which an application isrunning to display feedback on a display screen of computing device 303.Step 604 can be looped continuously with steps 601, 602, and 603 inreal-time. Alternatively, step 604 can occur when the application oncomputing device 303 is accessed, at which point sensor device 301synchronizes data with computing device 303.

Movement State Determination

As discussed with reference to step 602, microprocessor 403 determinesthe movement state of a user. To do this, microprocessor 403 analyzestri-axial acceleration data received from accelerometer 405 to determineuser movements along the x-, y-, and z-axes indicative of movementstates and/or transitions between the movement states. Theseaccelerometer data can be categorized by microprocessor 403 using astate machine with multiple movement states (e.g., running, walking,standing, sitting, jumping, squatting, and lying), as depicted in thestate diagram of FIG. 7.

Microprocessor 403 can determine the movement state of a user directlyusing a statistical model 702 (as detailed further herein with referenceto FIG. 10). In some instances (e.g., when a user is lying),microprocessor 403 can use a pelvic tilt angle model 704 (as detailedfurther herein) to directly determine a user's movement state. Pelvictilt angle model 704 is shown in the figure at only one state transition(between lying and running). One of skill in the art will understand,however, that this model can be used to determine transitions betweenlying and other movement states, as well as to directly determine alying state.

Microprocessor 403 can also determine the user's movement stateindirectly (using a library comparison model or a rule-based model) byanalyzing accelerometer data to identify a state transition (indicatedby linear arrows in the figure) indicative of a known transition betweentwo known movement states. As an example, if microprocessor 403identifies a state transition as a sitting-to-standing transition,microprocessor 403 determines that the user is moving into a standingstate. If instead microprocessor 403 identifies a state transition as asitting-to-lying transition, microprocessor 403 determines that the useris moving into a lying state. Microprocessor 403 assumes that the userremains in the determined movement state until a new state transition orstate is detected. To identify state transitions, microprocessor 403analyzes the accelerometer data using a library comparison model 703 (asdetailed further herein), a rule-based model 701 (as detailed furtherherein with reference to FIG. 9), and/or a pelvic tilt angle model 704.Although library comparison model 703 and rule-based model 701 aredepicted in the figure at only one state transition (e.g., betweensitting and lying and between standing and sitting, respectively) one ofskill in the art will understand that these models can be used fortransitions between any two movement states.

State transitions have unique acceleration patterns. As shown in a graphof exemplar data in FIG. 8, as a user transitions from a standing to asitting position, there is an initial moderate acceleration downward (a)along the y-axis as well as (c) along the z-axis, followed by (b) animmediate acceleration upward in the y-axis as the user hits the chair,and (d) a gradual increase in acceleration values as the user leans backinto the chair. Moderate acceleration is seen along the x-axis and peaks(e) when the user hits the chair.

Although some aspects of the state transitions may be individual- and/orsituation-specific (e.g., speed of the transition), tri-axialacceleration patterns of state transitions are similar across differentindividuals. Additionally, the tri-axial acceleration pattern for atransition from a first movement state (e.g., standing) to a secondmovement state B (e.g., sitting) is distinguishable from theacceleration pattern for a reverse transition from the second movementstate (e.g., sitting) to the first movement state (e.g., standing) eventhough the tri-axial acceleration patterns for each state transition(e.g., standing to sitting transition) are similar across individuals.Thus, these tri-axial acceleration patterns can be used bymicroprocessor 403 to detect state transitions and thereby identify themovement states contributing to the state transition (i.e., a previousmovement state and a current movement state). Once the current movementstate is identified, a user is considered to remain within that movementstate until a new state or state transition is detected.

Library Comparison Model.

With the library comparison model, state transitions are identified bycomparing continuous real-time acceleration signals from accelerometer405 with a movement library comprised of known state transitionpatterns. Known transition patterns are accelerometer signals accruedover time from multiple users for known transitions between two discretestates (e.g., accelerometer signals indicative of a transition betweenstanding and sitting, between standing and walking, between walking andrunning, etc.). If a match is found between the real-time signal and aknown transition pattern in the movement library, then the statetransition is identified and stored in memory.

In one embodiment, real-time accelerometer data are stretched orcompressed in time before comparison to the movement library in order tomimic a state transition being made more rapidly or less rapidly. As anexample, a user slowly standing from a sitting position over, e.g., a1-second time period, generates continuous real-time accelerometer datato be compared to transition patterns in the movement library. The1-second period data are also compressed (e.g., to occur over a0.5-second time period) and stretched (e.g., to occur over a 2-secondtime period). Thus, multiple data samples (e.g., 3 in this example) arecompared to the movement library in order to find a match to a knownstate transition. Stated differently, if the real-time motion data arefaster but qualitatively equivalent to a transition pattern in themovement library, stretching the real-time signal over a longer timeperiod will yield a match with the transition signal in the movementlibrary, thus allowing the state transition to be identified.

The library of patterns is built from historical measurements of users(with sensor device 301 attached) transitioning between movement states(e.g. standing up, sitting down, etc.).

Transitions involving a lying state, however, are determined bycomputing a pelvic tilt angle orientation (discussed elsewhere herein).For these transitions, if microprocessor 403 detects that the pelvictilt angle orientation of accelerometer 405 meets a thresholdindicating, e.g., an orientation less than 30 degrees with respect tothe ground, then microprocessor 403 determines that the user is lyingdown. If a lying state is determined, the directionality (i.e., a userlying on his stomach vs. lying on his back) is determined by theorientation of the z-axis.

Microprocessor 403 stores a current state (i.e., the state at which thedetermined transition was completed, e.g., sitting if the determinedtransition was from standing to sitting) in memory with a time stamp.The current state is assumed to persist until microprocessor 403receives real-time data from accelerometer 405 that is determined bymicroprocessor 403 to indicate another state transition (e.g., fromsitting to lying). The time-stamped current state and time-stamped statetransitions are stored in memory, and can be transferred at some time tocomputing device 303 when sensor device 301 is synchronized withcomputing device 303.

Rule-Based Model.

State transitions can also be determined by using a rule-based model.The rule-based model uses gliding sample windows to detect extrema ofvarious different time-domain features (e.g., x, y, or z) and thenanalyzes the sequence of x, y, and z extrema and their properties (e.g.,acceleration amplitude) with heuristic rules to look for statetransitions. In one embodiment, microprocessor 403 analyzes datastreamed in real-time (e.g., at 32 Hz) from accelerometer 405 using asliding sample window (e.g., a 32-sample window such that a first windowcontains accelerometer data points in 1-32, a second window containsdata points in frames 2-33, a third window contains data points inframes 3-34, etc.). Thus, data for a certain percentage of a movement ormovement transition is captured within each window. Microprocessor 403analyzes the streamed accelerometer data in sample windows to identify amaximum y-value (“y_(max)”, indicative of maximal verticalacceleration), a minimum z-value (“z_(min)”, indicative of minimalforwards/backwards acceleration), and a quiescence signal value(“y_(quiet)”, indicative of minimal acceleration in the y-axis). Asthese values are identified, they are fed into a 3-element queue. Whenthe queue contains 3 values that match a pattern associated with a knowntransition (i.e., between known movement states), microprocessor 403 canidentify the movement state. A flowchart detailing steps of a method todetermine movement state using this digital signal processing modelaccording to one embodiment is presented in FIG. 9.

In step 901, microprocessor 403 determines whether y_(t)=y_(max).Microprocessor 403 makes this determination by comparing the currenty-value y_(t) to all y-values of an immediately preceding window w_(p)and to all y-values of an immediately succeeding window w_(s).Mathematically, if y_(p)<y_(t) for all y_(p) in w_(p) and if y_(t)>y_(s)for all y_(s) in w_(s), then y_(t)=y_(max)

If y=y_(max), then, in step 905, an indicator of y_(max) (e.g., anominal tag, e.g., “A” or “max”), an accelerometer measurementassociated with that y_(max), and a time stamp associated with thataccelerometer measurement, are output as one value (with features ofindicator, gravity, time) to the 3-element queue.

If y≠y_(max), then, in step 902, microprocessor 403 determines whetherz_(t)=z_(min). Microprocessor 403 makes this determination by comparingthe z-value of the current z-value z_(t) to all z-values of theimmediately preceding window w_(p) and to all z-values of theimmediately succeeding window w_(s). Mathematically, if z_(p)>z_(t) andif z_(t)<z_(s), then z_(t)=z_(min).

If z=z_(min), then, in step 906, an indicator of z_(min)(e.g., a nominaltag, e.g., “B” or “min”), an accelerometer measurement associated withthat z_(min), and a time stamp associated with that accelerometermeasurement, are output as one value (with features of Indicator,gravity, time) to the 3-element queue.

If z≠z_(min), then, in step 903, microprocessor 403 determines whethery_(quiet) exists. Microprocessor 403 makes this determination bycomparing local y-values (i.e., values of y within the current window).One of skill in the art will understand that the sample window used todetermine y_(quiet) need not contain the same number of samples as thesample window used to calculate y_(max) and z_(min). If the maximumy-value within the current window is close to the minimum y-value withinthe current window (e.g., +/−5 where 250 is 1G, which is to say, a yrange is 0.02 G's), then the midpoint between the maximum and minimumy-values is determined to be y_(quiet).

If y_(quiet) exists within a current window, then, in step 907, anindicator of y_(quiet) (e.g., a nominal tag, e.g., “C” or “quiet”), anaccelerometer measurement associated with that y_(quiet), and a timestamp associated with that accelerometer measurement, are output as onevalue (with features of indicator, gravity, time) to the 3-elementqueue.

If y_(quiet) does not exist within the current window, then, in step904, microprocessor 403 loops back to step 901, a next window becomes anew current window, and microprocessor 403 begins analyzing y-/z-valuesin the new current window.

In step 908, microprocessor 403 determines whether the pattern of valuesin the queue (the “queue pattern”) matches a pattern associated with aknown transition between movement states. For example, if the queuepattern is ordered as “min-max-quiet” (or “B-A-C”), the queue matches aknown pattern for a transition from a standing to a sitting state. Or,if the queue pattern is ordered as “quiet-max-min” (or “C-A-B”), thequeue matches a known pattern for a transition from a sitting to astanding state.

If, in step 908, microprocessor 403 determines that the queue pattern isa known state transition pattern, then, in step 909, microprocessor 403identifies a new movement state associated with that state transitionpattern and stores that information with a time-stamp in memory.

If microprocessor 403 determines that the queue pattern is not a knownstate transition pattern (in step 909), then, in step 904,microprocessor 403 loops back to step 901, a next window becomes a newcurrent window, and microprocessor 403 begins analyzing y-/z-values inthe new current window.

One of skill in the art will recognize that steps 901, 902, and 903 neednot be performed in the order described herein, but may be performedsimultaneously, in reverse order, or otherwise. One of skill in the artwill further understand that as new values are placed into the queue,older values are displaced (pushed out) from the queue. Thus, thepattern within the queue changes each time a newly identified y_(max),z_(min), or y_(quiet) is identified by microprocessor 403.

Tilt Angle Model.

Tri-axial acceleration patterns can distinguish transitions involvingmost states, but are less robust for determining transitions to and/orfrom a lying state. These transitions (involving a lying and/or sittingstate) can be better distinguished using a pelvic tilt angle model. Ifmicroprocessor 403 detects that the pelvic tilt angle (as measured byorientation of accelerometer 405) meets a threshold indicating, e.g., anorientation of less than 30 degrees, microprocessor 403 determines thatthe user is lying down. If a lying state is determined, thedirectionality (i.e., a user lying on his stomach vs. lying on his back)is determined by the orientation of the z and x axes.

Statistical Model.

As discussed with reference to FIG. 7, microprocessor 403 can determinethe movement state of a user directly using a statistical model 702.Because microprocessor 403 runs the statistical model continuously onincoming data from accelerometer 405 (as indicated by circular arrowsnext to each state in the figure), microprocessor 403 can determine whenthe user moves from one state (e.g., sitting) into a different state(e.g., standing). A flowchart of a method to determine movement statethrough a statistical model according to one embodiment is presented inFIG. 10, although the details of each step are discussed in furtherdetail with respect to exemplar data.

In step 1001, microprocessor 403 normalizes accelerometer data.

In step 1002, microprocessor 403 computes feature vectors to reducedimensionality of the accelerometer data.

In step 1003, microprocessor 403 quantizes the feature vectors and usesclustering techniques to assign the quantized feature vectors toGaussian distributions.

In step 1004, microprocessor 403 acquires estimated probabilityhistograms for discrete movement states.

In step 1005, microprocessor 403 determines the movement state bycomparing probabilities of different movement states.

These steps of the statistical model will now be explained in greaterdetail by way of example.

In real-time, microprocessor 403 retrieves accelerometer measurements at32 Hz (i.e., 32 three-dimensional vectors (x, y, z) per second). In thefollowing example, three exemplar vectors (e.g., measured vectors v1,v2, and v3 of 32 vectors over frames 1, 2, and 3 respectively of a onesecond, 32-frame period) are used, representing about a tenth of asecond of data recorded from a walking person.

Accelerometer 405 is a 10-bit accelerometer with output values rangingfrom −512 to 511. Gravity corresponds to a reading of 248. Thus, if theuser remains stationary, with 0 movement in two axes, the third axisshows a reading of 248. The exemplar measured vectors v1, v2, and v3are, respectively,

$\underset{v\; 1}{\begin{pmatrix}271 \\32 \\{- 148}\end{pmatrix}},\underset{v\; 2}{\begin{pmatrix}310 \\{- 87} \\{- 100}\end{pmatrix}},\underset{v\; 3}{\begin{pmatrix}287 \\{- 76} \\20\end{pmatrix}}$

Normalization of Accelerometer Data (step 1001).

In step 1001, incoming vectors 1, 2, and 3 are rotated to normalize theaccelerometer orientation. Two rotations (R₀ and R₁) are performedconcurrently to find orientation matrices R₀ (representing a rotationaround the z-axis) and R₁ (representing a rotation around the y-axis)such that

$\begin{pmatrix}\overset{\sim}{x} \\\overset{\sim}{y} \\\overset{\sim}{z}\end{pmatrix} = {R_{1}{R_{0}\begin{pmatrix}x \\y \\z\end{pmatrix}}}$

These orientation matrices (R₀ and R₁) are computed at least once for auser and then stored in memory so they need not be recomputed each timemeasurements are received from accelerometer 405.

Using the exemplar vectors,

$R_{0} = \begin{pmatrix}{- 0.0166} & {- 0.9999} & 0 \\0.9703 & {- 0.0161} & {- 0.2414} \\0.2414 & {- 0.0040} & {- 0.9704}\end{pmatrix}$ and $R_{1} = \begin{pmatrix}{- 0.9974} & 0 & 0.0724 \\0 & 1 & 0 \\0.0724 & 0 & {- 0.9974}\end{pmatrix}$

Sending the three exemplar vectors through this transform results inrotated vectors

$\underset{v\; 1}{\begin{pmatrix}30.74 \\298.16 \\80.78\end{pmatrix}},\underset{v\; 2}{\begin{pmatrix}83.20 \\326.33 \\15.89\end{pmatrix}},\underset{v\; 3}{\begin{pmatrix}64.59 \\274.87 \\93.90\end{pmatrix}}$

Computation of Feature Vectors (step 1002).

Even after normalizing accelerometer orientation, the rawthree-dimensional vectors can be improved for activity recognition.Thus, in step 1002, microprocessor 403 computes feature vectors from theraw three-dimensional normalized vectors. Because microprocessor 403 haslimited memory (e.g., 8 KB of RAM) and limited computing power (e.g., 16bit, 20 MHz) in one embodiment, a set of two features (α, β) arecalculated for use in lieu of these typical transformations:

α_(t)=sgn(z)*√{square root over (x ² +z ²)}

and

β_(t)=α_(t)−α_(t-1)

where t is a time index.

Using this feature set (α, β) reduces dimensionality while addingmotion-related information because β measures the difference between thecurrent and previous value of α.

The three example vectors are now reduced to:

$\underset{v\; 1}{\begin{pmatrix}86.43 \\0\end{pmatrix}},\underset{v\; 2}{\begin{pmatrix}84.70 \\{- 1.73}\end{pmatrix}},\underset{v\; 3}{\begin{pmatrix}{- 113.97} \\{- 198.68}\end{pmatrix}}$

Note that β₀=0 by default because no previous value exists (before v1 atthe first time point) from which to compute a difference.

Quantization of Feature Vectors (step 1003).

After the feature vectors are computed, then in step 1003, the featurevectors are put through a process called vector quantization in whichthe vectors are restricted to discrete values (rather than a continuousset of values such as raw accelerometer data).

Incoming, rotated measurement vectors are assigned to an inventory ofGaussian distributions (a multivariate Gaussian mixture model) that thevector best matches. Density values of the vectors are used as a metricto identify the best matching distribution. Once a distribution isidentified that best matches the incoming rotated measurement vector,the original vector is ignored, and microprocessor 403 instead uses anindex of the best-matched distribution for remaining calculations.

In other words, the sequence of (exemplar) vectors is mapped to asequence of integers:

ν₁, . . . ,ν_(n)

c ₁ , . . . ,c _(n)

where each c_(i) is an integer representing the Gaussian distributiong_(c) _(i) .

In one embodiment, the multivariate Gaussian mixture model is derivedwith a clustering procedure (e.g., using an mclust package of an Rplatform analytics program (TIBCO Software Inc., Palo Alto, Calif.)).Input for the clustering procedure is a large collection of featurevectors

$\quad\begin{pmatrix}\alpha \\\beta\end{pmatrix}$

in a comma-delimited file. The R commands to read the file and performthe clustering are:

> data <- read.csv(file=“myfile.csv”,head=TRUE,sep=“,”) > mycols <-data[,c(‘XZ_ROT’,‘DELTA_XZ_ROT’)] >mc<-Mclust(mycols,G=1:100,modelNames=“EVI”) >sink(“c:/mydir/mymixture.txt”, append=FALSE, split=FALSE) >mc$parameters > sink( )

The output of this procedure is a set of, at most, 100 normaldistributions, each with an a priori probability. The overall densityvalue of a vector ν is given by

f(ν)=Σπ_(c) f(ν,μ_(c),Σ_(c))

where c is an index of looping through all the clusters, π_(c) is an apriori probability of a cluster c, μ_(c) defines a cluster center, andΣ_(c) is a covariance matrix. The term f(ν, μ_(c), Σ_(c)) denotes amultivariate Gaussian with parameters μ_(c) and Σ_(c). The R platformprogram assumes an equal volume independent (“EVI”) model, which meansthat covariance matrices Σ_(c) are diagonal and of equal volume. In atwo-dimensional case with k as a cluster index,

Σ_(k)=(σ_(k)(α),σ_(k)(β)).

The equal volume assumption means that

σ_(k)(α)σ_(k)(β)=σ_(i)(α)σ_(i)(β) for all i,k.

(As is known, this expression is called volume in analogy to the area(or volume) of an ellipse whose major and minor axis are σ_(k)(α) andσ_(k)(β).)

Exemplar vector ν (e.g., v₁, v₂, or v₃) is assigned to cluster c if

π_(c) f(ν,μ_(c),Σ_(c))≧π_(d) f(ν,μ_(d),Σ_(d)) for all clusters d.

Due to the EVI assumption, it is not necessary to compute a value ofnormal distributions for all d (e.g., ν, μ_(d), Σ_(d)). Instead, aMahalanobis distance can be used to arrive at the exact same result:

${{2\mspace{11mu} \ln \mspace{11mu} \pi_{c}} + \frac{\left( {\alpha - {\mu_{c}(\alpha)}} \right)^{2}}{\sigma_{c}^{2}(\alpha)} + \frac{\left( {\beta - {\mu_{c}(\beta)}} \right)^{2}}{\sigma_{c}^{2}(\beta)}} \leq {{2\mspace{11mu} \ln \mspace{11mu} \pi_{d}} + \frac{\left( {\alpha - {\mu_{d}(\alpha)}} \right)^{2}}{\sigma_{d}^{2}(\alpha)} + \frac{\left( {\beta - {\mu_{d}(\beta)}} \right)^{2}}{\sigma_{d}^{2}(\beta)}}$

for all clusters d.

The clusters can be visualized graphically by writing a program thatgenerates samples from the Gaussian distributions with the help of arandom number generator.

When this is done, the mixture model consists of each Gaussiandistribution represented by a cloud with an a priori probability. Themixture model for the exemplar data clusters are shown in FIG. 11.Arrows in the figure represent a location of the first two exemplarvectors (v₁, v₂) which are so closely located as to be indistinguishablein the figure and the third exemplar vector (v₃). Thus, vectorquantization reduces exemplar input measurements to a sequence 43, 43,17—that is, to the integer representing the distribution most closelymatched by each exemplar vector.

Estimation of Distribution Histograms for Discrete Movement States (Step1004).

In step 1004, microprocessor 403 acquires estimated distributionhistograms for discrete movement states. In one embodiment,microprocessor 403 estimates the distribution histograms, while in otherembodiments microprocessor 403 retrieves the distribution histogramsfrom computing device 303 or via computing device 303 from cloud server305. Regardless, the distribution histograms are estimated as follows.Given an integer sequence c₁, . . . , c_(n) (representing Gaussiandistributions 1 through n) of quantized accelerometer readings (i.e.,representing v₁, . . . , v_(n)), and knowing that the sequence wasgenerated while performing a known state (e.g. walking), a frequency ofeach cluster for that activity can be estimated to generate a discreteprobability histogram of the clusters for each activity. Thesehistograms are evaluated to determine the user's movement state. In eachhistogram, the probability of the clusters for that activity sum to 1.

For the determined integer sequence of the quantized vectors (e.g., theexemplar sequence 43, 43, 17), probabilities are determined from clusterhistograms for the movement states. An estimated exemplar clusterhistogram for a walking state is presented in FIG. 12 and for a runningstate in FIG. 13. Probabilities from these histograms are used todetermine a probability for each integer of the exemplar sequence.

To determine a probability that a user is in a particular movementstate, microprocessor 403 multiplies the probabilities (from the clusterhistogram for the particular state) of the sequence integers (43, 43,17). In FIGS. 12 and 13, the probabilities for the sequence integers arecircles indicated with arrows. Thus, the probability that the user is ina walking state is:

P(walk)=P _(w)(43)P _(w)(43)P _(w)(17)=0.0168*0.0168*0.0043=1.214×10⁻⁶

Likewise, the probability that the user is in a running state is:

P(run)=P _(r)(43)P _(r)(43)P _(r)(17)=0.0216*0.0216*0.0129=6.02×10⁻⁶

Comparison of Distribution Histograms for Discrete Movement States (Step1005).

In step 1005, microprocessor 403 determines a user's current movementstate by comparing the separate probabilities for the integer sequencein discrete movement states to determine which movement state is mostlikely. In the exemplar data, the probability that the user is in arunning state (6.02×10⁻⁶) is greater than the probability that the useris in a walking states (1.214×10⁻⁶). Thus, running is the determinedstate indicated by the exemplar data.

After evaluating the determined statistical model, microprocessor 403stores the determined state in memory and loops back to step 1001 tobegin the process again by normalizing a next set of accelerometer datavectors. By continually performing this process, microprocessor 403 canidentify when the user has moved into a new movement state.

For longer sequences, however, three issues arise:

-   -   How to prevent numeric underflow which can occur because        multiplying probabilities results in ever smaller numbers;    -   How to adapt probabilities over time since a new state (e.g.,        walking) must be detectable even after a user has been in a        different state (e.g., running) for a long period of time; and    -   How to adjust for clusters with a probability of 0 for an        activity since such clusters necessarily drive an overall        multiplicative probability for the activity to zero (from which        no recovery is possible).

Underflow Issue.

In one embodiment, to prevent the numeric underflow problem, naturallogarithms of the probabilities are used instead of the probabilitiesthemselves. Conveniently, this adjustment also converts multiplicativeprobability calculations into additive calculations to generate logprobability scores (and also saves some processing time). For theexemplar data, then,

ln P(walk)=ln P _(w)(43)+ln P _(w)(43)+P _(w)(17)=−5.54−5.54−5.45=−16.53

ln P(run)=ln P _(r)(43)+ln P _(r)(43)+P _(r)(17)=−3.84−3.84−5.45=−13.13

As with linear probabilities as discussed elsewhere herein,microprocessor 403 compares the log probabilities to determine that,since the log probability of running (−13.13) is greater than that ofwalking (−16.53), running is the more likely movement state. As Papproaches 0, log probability scores (i.e., ln(P)) approach negativeinfinity, but very slowly, so the calculated log probabilities are notas small as those calculated as linear probabilities.

Adaptation Over Time Issue.

Probabilities for different activity states should not drift apart toomuch otherwise an hour of one state (e.g., walking) would be necessaryto compensate for an hour of a different state (e.g., running). To avoidthis problem, in one embodiment, the log of the determined activity(e.g., running in the exemplar data) is scaled to some fixed target logprobability ln(P_(T)), and all other log probabilities are multiplied bythat scaling adjustment factor. This scaling adjustment occurs duringevery data processing frame (i.e. 32 times a second). The target logprobability also provides a mechanism to adjust the responsiveness ofthe classifier. For example, if ln(P_(T)) is close to 0 (i.e., P_(T) isclose to 1), the classifier will react very quickly to changes. Ifln(P_(T)) is set to more negative values, the classifier will react moreslowly. By experimentation, ln(P_(T))=−50 turns out to be a good value.

Using the exemplar data, the determined log probability score for thefirst frame of the determined running state is −3.84, so the scalingadjustment factor is 50/3.84=13.02. The first frame of the walk state is−5.54, so ln P(walk) gets scaled with the scaling adjustment factor to−5.54*13.02=−72.13.

Zero Probabilities Issue.

In one embodiment, probabilities in the model distributions are notallowed to be zero. Any probability below 1E-5 is replaced by 1E-5. Interms of logs, this convention means that no ln(P(c)) can be below−11.51.

Movement State Model Conflict Resolution.

For most movement states, the statistical and rule-based models areequally robust in identifying the movement state and both are superiorto the library comparison model. In some instances, however, themovement state determined by microprocessor 403 varies depending on themodel used (i.e., for the same data, a movement state determined usingthe statistical model is not the same as the movement state determinedusing the rule-based model). In these instances, microprocessor 403defaults to a conflict resolution protocol.

Conflict resolution between statistical model determinations andrule-based model determinations depends on the user's movement state.For example, because sitting and standing signatures look similar (i.e.,accelerometer readings for sitting and standing without moving aresubstantially the same), the statistical model is not ideal fordetermining the difference between sitting and standing states. Thus,for sitting or standing movement states, the rule-based model istypically used to identify transitions between sitting and standingstates (as discussed elsewhere herein) and those determinationstypically override the results of the statistical state recognitionaccordingly. For the other movement states (e.g., lying, walking andrunning), statistical model determinations of the user's movement stateare given priority over the rule-based model determinations of theuser's movement state.

Posture Quality Determination

Once microprocessor 403 has determined a user's movement state by usingone or more of the models (library comparison model, rule-based model,tilt angle model, and/or statistical model), microprocessor 403determines the user's posture quality.

Microprocessor 403 uses accelerometer data to determine a user's posturequality (i.e., how well the user is balanced within a movement state,e.g., whether and how well the user's posture matches a desirable “goodposture” for the determined movement state) over time. Microprocessor403 makes this determination based on an average pelvic tilt angle(relative to the ground) derived from accelerometer data sampled overtime (e.g., 750 times over 25 seconds). The tilt angle is measured, asis known, by calculating an angle between a sensing axis and the gravityvector. See, e.g., M. Clifford & L Gomez (2005), “Measuring Tilt withLow-g Accelerometers”, in Freescale Semiconductor (Application Note). Anaverage tilt angle can be determined at any point after theaccelerometer data have been normalized. Once the average pelvic tiltangle has been calculated, microprocessor 403 stores the average pelvictilt angle with a time stamp in memory. An updated pelvic tilt angle iscalculated at established time intervals when a user remains in adetermined movement state and/or after the user transitions to a newmovement state.

Referring again to FIG. 2, good posture (shown in diagram 201) isdefined as a posture with a neutral spine and a slightly antevertedpelvis such that the tip of the pelvis is tilted slightly forward. Whenthe pelvis is slightly anteverted, the vertebrae stack up on each otherand carry the weight of the body. An overly anteverted pelvis (shown indiagram 202) can cause an over-extension of the lumbar spine withconsequent back health issues. When the pelvis is retroverted (tip ofthe pelvis is tilted back) the spine puts pressure on the lower backmuscles to carry the weight of the upper back. An overly retrovertedpelvis (shown in diagram 203) causes flexion in the lumbar spine whichalso can lead to back health issues.

A scale of good posture is determined by the tilt angle of the pelvis.Different body types may yield slightly different pelvic tilt anglereadings, but correct calibration of accelerometer 405 when sensordevice 301 is attached alleviates this issue. Good posture isbenchmarked off of each individual user's average pelvic angle capturedwhen walking.

When the user is standing with sensor device 301 attached at or aroundL4-L5 vertebrae, good posture is defined by a pelvic tilt angle readingof +/−5° compared to the average pelvic tilt angle captured whilewalking. At these angles, the pelvis is in a neutral position, onaverage. When the pelvis rotates backwards into a retroverted positionsuch that the lumbar spine is flexed, the tilt angle increases beyond 5°greater than the average pelvic angle captured while walking. When thepelvis rotates forward into an overly anteverted position, the tiltangle decreases beyond 5′ below the average pelvic angle captured whilewalking. Thus, poor posture is defined by pelvic tilt angles less thanor greater than 5° of the average pelvic angle captured while walking.When the user is sitting, good posture is defined as a pelvic tilt angleof +/−10° of the average pelvic angle captured while walking. Poorsitting posture is defined as a pelvic tilt angle greater than +/−10° ofthe average pelvic angle captured while walking.

When the user is standing with sensor device 301 attached at or aroundS1-S2 vertebrae, good posture is also defined by a pelvic tilt anglereading of, e.g., +/−5° of the average pelvic angle captured whilewalking, with respect to the ground. At these angles, the pelvis is, onaverage, in a neutral position. When the pelvis rotates backwards into aretroverted position such that the lumbar spine is flexed, the tiltangle increases beyond the 5° of the average pelvic tilt angle capturedwhile walking. When the pelvis rotates forward into an overly antevertedposition, the tilt angle decreases beyond 5° below the average pelvictilt angle while walking. Thus, poor posture is defined by pelvic tiltangles of less than or greater than, e.g., 5° of the average pelvic tiltangle captured while walking. When the user is sitting, good posture isdefined as a pelvic tilt angle of, e.g., +/−10° of the average pelvictilt angle while walking. Poor sitting posture is defined as a pelvictilt angle of less than or greater than, e.g., 10° of the average pelvictilt angle while walking.

If the average pelvic tilt angle while walking does not give an accurateenough determination of the user's good sitting and/or standing posture,then the user can start a self-calibration process. This entailswatching an educational video, followed by a self-calibration for thesitting and standing postures using computing device 303.Self-calibration using a computing device involves getting into a goodseated or standing position, beginning the calibration process oncomputing device 303, and holding the good posture for several momentswhile sensor device 301 averages the positional data of the goodpostural position. This process is used for either or both sitting andstanding positions.

Biofeedback Based on User's Postural Description

Postural feedback is provided to the user in real-time via actuator 406,and/or through an application running on computing device 303. Posturalfeedback provided through the application running on computing device303 can be provided in real-time, or a later time when computing device303 is synchronized with sensor device 301.

In one embodiment, an animated graphic in the application running oncomputing device 303 is used to convey postural feedback to the user.The animated graphic is an avatar, an abstracted real-time animatedcharacter that reflects the movement and posture of the user inreal-time. If the user sits with good posture, the avatar sits with goodposture. If the user slouches, the avatar slouches. When the user standsup, the avatar stands up. When a user begins walking, the avatar walks.When the user runs, the avatar runs. When the user lies down, the avataralso lies down.

In a further embodiment indicating the user's posture, the avatardisplays emotive feedback by, for example, assuming facial expressionsand colors associated with good or bad posture. As an example shown inFIG. 14, when the user has good posture in a movement state (e.g.,standing upright with good posture or sitting with good posture), avatar1401 mimics the good posture in that movement state (e.g., whilestanding or sitting) as well, and couples the good posture with a smileon his face and a green body (not shown) to symbolize correct (“good”)posture. When the user has poor posture in a movement state (e.g.,slouches while sitting), avatar 1402 likewise displays poor posture inthat movement state (e.g., slouches while sitting), and couples the poorposture with a less happy face and an orange body (not shown) as awarning that posture is not optimal. Over time, as a user improves hisposture and movement, the avatar's appearance improves (e.g., largermuscles, certain clothing) to tangibly display the user's improvement.As an example, as a user's posture improves, avatar 1403 begins to havelarger muscles and/or wears more attractive clothing or accessories(e.g., a crown).

This graphical display of information has many uses beyond relayingdata. For instance, the avatar can be used to communicate with otherfriends and family also using a sensor device 301, thereby facilitatinginterpersonal communication, goal tracking, and games. For example,multiple users can share their avatars with each other and view all ofthe avatars on display screens of their respective computing devices303. Games and challenges can also be facilitated through this newexperience. For example, users can challenge each other to a posturegame in which each user aspires to have the best posture for a specificamount of time. Each user's corresponding avatar will reflect hisprogress towards this challenge in real-time on the other users'computing devices. Users can also cooperatively help each other withgoal-oriented games, as certain gestures may enable the avatars tonavigate through mazes or capture a certain number of objects with theaim of attaining a certain goal. In addition, this visualization ofanother user's avatar as well as one's own avatar allows for friends andfamily to interact with each other through virtual real-timevisualization, thereby allowing the users to comment on biomechanics ofthe other users' postures in real-time.

Communication of user data and user avatars to other users isaccomplished by transmitting movement state and posture quality tocomputing devices 303 via wireless communication module 404 and cloudserver 305. The application running on a first user's computing device303 interprets the movement state and posture quality incoming from asecond user's computing device 303 via cloud server 305 as a specificposition and animates an avatar representing the second user based onnew position data points.

Customization of Sensor Device

The user can control sensor device settings (e.g., sensory feedbacksettings) through the application running on computing device 303. Forexample, a user can activate calibration of personalized posture andmovement, can change actuator settings such duration and/or intensity offeedback actuation (e.g., duration and/or intensity of vibration) and/orduration of in a poor posture before feedback is triggered, or turn offsensory feedback.

If the user activates calibration of personalized posture and movement,the user can record a series of personalized stationary postures, pelvictilt angles, and/or state transition signals for later use inquantifying good and bad posture. Recorded personalized statetransitions are stored in the movement library and used for comparisonagainst live acceleration data to better detect various movementtransitions.

In addition, the sensitivity to trigger biofeedback can change over timegiven the user's improvement in posture. For example, as a higherpercentage of correct posture positions are detected for the user overtime, time to trigger the biofeedback can decrease (i.e., there is aquicker trigger of the vibratory feedback as a user's performanceimproves).

When the user desires to change sensor device settings or calibrations,these changes are transmitted via wireless communication module 404 tosensor device 301 and stored within memory in microprocessor 403 untilor unless changed again.

Behavioral patterns analyzed from user data in cloud server 305 can alsobe used to calibrate sensor device settings and performance. Forexample, responses to biofeedback can be analyzed. If the user does notrespond readily to feedback, the system can increase the feedbackactuation intensity or decrease the feedback frequency. For example,after behavioral analysis in cloud server 305, a command can be sent tocomputing device 303 to change sensor device settings. In addition, userdata can be stored in cloud server 305 can be used to tailor thestatistical model and/or the rule-based model, as well as thebiofeedback model to that user.

Finally, personalized recommendations such as tips, exercises,stretches, and goal settings, can be provided to the user based onbehavior analysis of the user's behavior over time. Behavior analysiscan consist of reviewing and combining the user's personal movementdata, the user's history of reaction to biofeedback, and/or the user'susage patterns and desire to change the settings of the real timebiofeedback and other settings.

The user may have a personal data hub on cloud server 305 which cancontain summarized data of user's posture, goals and activity breakdownavailable for viewing on computing device 303. Movement and behaviorpatterns of the user are analyzed in cloud server 305 to helppersonalize behavior of sensor device 301 and to tailor behavioralrecommendations to the user. For example, various movements, exercises,and stretches may be analyzed to find correlations with the increase ordecrease of back pain as discussed elsewhere herein.

In addition to using the user's personal data to provide feedback tothat user, personal data from multiple users can be combined foranalysis in cloud server 305 and used to provide feedback. For example,data in cloud server 305 can be mined to study movements correlated toincreasing or decreasing lower back pain. Such an analysis can yield newpersonalized regimens and treatments for managing back pain in differentpopulation groups. As another example, movement signatures of peoplewith varying degrees of symptomatology (e.g., severe, moderate, and mildback pain) can be analyzed demographically. Correlations of data fromusers based on demographics, movement, and/or symptoms can be used toprovide personalized improvement plans and/or actionable goals that maylead to alleviation of back pain, increased confidence, and a quickermethod to more toned core muscles. Such personalized plans can beautomatically implemented by the sensor itself, or can be controlled bythe user such that the user can implement portions of the plan asdesired. Behavioral data (e.g., responses to biofeedback actuations) canalso be analyzed to personalize sensor device behavior so as to providefeedback to the user during optimal instances in which the user may bemore motivated or open to adjusting posture position.

The disclosed method and apparatus has been explained above withreference to several embodiments. Other embodiments will be apparent tothose skilled in the art in light of this disclosure. Certain aspects ofthe described method and apparatus may readily be implemented usingconfigurations other than those described in the embodiments above, orin conjunction with elements other than those described above. Forexample, although embodiments of the sensor device described hereindiscuss placement of the sensor device on the back, the sensor devicecan be placed on other parts of the body (e.g., neck) as well as onvarious equipment such as chairs, golf clubs, tennis racquets, etc.

Further, it should also be appreciated that the described method andapparatus can be implemented in numerous ways, including as a process,an apparatus, or a system. The methods described herein may beimplemented by program instructions for instructing a processor toperform such methods, and such instructions recorded on a computerreadable storage medium such as a hard disk drive, floppy disk, opticaldisc such as a compact disc (CD) or digital versatile disc (DVD), flashmemory, etc., or a computer network wherein the program instructions aresent over optical or electronic communication links. It should be notedthat the order of the steps of the methods described herein may bealtered and still be within the scope of the disclosure.

It is to be understood that the examples given are for illustrativepurposes only and may be extended to other implementations andembodiments with different conventions and techniques. While a number ofembodiments are described, there is no intent to limit the disclosure tothe embodiment(s) disclosed herein. On the contrary, the intent is tocover all alternatives, modifications, and equivalents apparent to thosefamiliar with the art.

In the foregoing specification, the invention is described withreference to specific embodiments thereof, but those skilled in the artwill recognize that the invention is not limited thereto. Variousfeatures and aspects of the above-described invention may be usedindividually or jointly. Further, the invention can be utilized in anynumber of environments and applications beyond those described hereinwithout departing from the broader spirit and scope of thespecification. The specification and drawings are, accordingly, to beregarded as illustrative rather than restrictive. It will be recognizedthat the terms “comprising,” “including,” and “having,” as used herein,are specifically intended to be read as open-ended terms of art.

What is claimed is:
 1. A method of providing postural feedbackcomprising: receiving by a microprocessor at repeated intervals datafrom a tri-axial accelerometer, the microprocessor and the tri-axialaccelerometer comprising a sensor device attached to a user, the sensordevice further comprising memory, an actuator, and a power source;normalizing by the microprocessor the received accelerometer data;determining by the microprocessor a postural description of the userbased on the normalized received accelerometer data; and triggering bythe microprocessor the actuator to output sensory feedback based on thepostural description of the user.
 2. The method of claim 1 furthercomprising transmitting across a wireless electronic communicationconnection to a computing device the postural description of the userand the normalized received accelerometer data.
 3. The method of claim 1wherein the postural description comprises a current movement state. 4.The method of claim 3 wherein determining the postural description basedon the normalized received accelerometer data comprises: computingfeature vectors for the normalized received accelerometer data;quantizing the feature vectors; assigning the feature vectors toGaussian distributions; acquiring distribution histograms for multiplemovement states; and comparing the distribution histograms to determinethe current movement state from the multiple movement states.
 5. Themethod of claim 1 wherein determining the postural description based onthe normalized received accelerometer data comprises calculating apelvic tilt angle of the user.
 6. The method of claim 1 wherein thepostural description comprises a current posture quality.
 7. The methodof claim 1 wherein the postural description comprises a current movementstate and a current posture quality.
 8. The method of claim 1 wherein arule-based model is used to determine the postural description of theuser based on the normalized received accelerometer data.
 9. The methodof claim 1 wherein a statistical model is used to determine the posturaldescription of the user based on the normalized received accelerometerdata.
 10. The method of claim 1 wherein a library comparison model isused to determine the postural description of the user based on thenormalized received accelerometer data.
 11. The method of claim 1wherein a pelvic tilt angle model is used to determine the posturaldescription of the user based on the normalized received accelerometerdata.
 12. A postural feedback apparatus comprising: a sensor deviceconfigured to be attached on a user, the sensor device comprising: atri-axial accelerometer; an actuator; and a microprocessor configured toreceive data from the tri-axial accelerometer about movement of theuser; normalize the received accelerometer data; determine a posturaldescription of the user based on the normalized received accelerometerdata; and trigger the actuator to output sensory feedback based on thepostural description of the user.
 13. The apparatus of claim 12 whereinthe sensor device further comprises a wireless communication module. 14.The apparatus of claim 12 wherein the microprocessor is furtherconfigured to communicate with the wireless communication module totransmit the postural description of the user and the normalizedreceived accelerometer data to a computing device.
 15. The apparatus ofclaim 12 wherein the postural description comprises a current movementstate.
 16. The apparatus of claim 15 wherein the microprocessorconfigured to determine the postural description of the user based onthe normalized received accelerometer data comprises the microprocessorconfigured to compute feature vectors for the normalized receivedaccelerometer data; quantize the feature vectors; assign the featurevectors to Gaussian distributions; acquire estimated distributionhistograms for multiple movement states; and compare the distributionhistograms to determine the current movement state from the multiplemovement states.
 17. The apparatus of claim 12 wherein the posturaldescription comprises a current posture quality.
 18. The apparatus ofclaim 12 wherein the postural description comprises a current movementstate and a current posture quality.
 19. The apparatus of claim 12wherein the microprocessor configured to determine the posturaldescription of the user based on the normalized received accelerometerdata uses a rule-based model.
 20. The apparatus of claim 12 wherein themicroprocessor configured to determine the postural description of theuser based on the normalized received accelerometer data uses astatistical model.
 21. The apparatus of claim 12 wherein themicroprocessor configured to determine the postural description of theuser based on the normalized received accelerometer data uses a librarycomparison model.
 22. The apparatus of claim 12 wherein themicroprocessor configured to determine the postural description of theuser based on the normalized received accelerometer data uses a pelvictilt angle model.
 23. A non-transitory computer readable medium havingstored thereupon computing instructions comprising: a code segment toreceive by a microprocessor at repeated intervals data from a tri-axialaccelerometer, the microprocessor and the tri-axial accelerometercomprising a sensor device attached to a user, the sensor device furthercomprising memory, an actuator, and a power source; a code segment tonormalize by the microprocessor the received accelerometer data; a codesegment to determine by the microprocessor a postural description of theuser based on the normalized received accelerometer data; and a codesegment to trigger by the microprocessor the actuator to output sensoryfeedback based on the postural description of the user.
 24. Thenon-transitory computer readable medium of claim 23 having storedthereupon computing instructions further comprising: a code segment totransmit from the microprocessor across an electronic communicationconnection to a computing device the postural description of the user tobe displayed on the computing device as an avatar with an appearance,the appearance of the avatar representing the postural description ofthe user.
 25. The non-transitory computer readable medium of claim 24having stored thereupon computing instructions further comprising: acode segment to transmit from the computing device across a networkconnection to a cloud server the postural description of the user, thecloud server using the postural description of the user to tailorbiofeedback for the user.